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ABSTRACT 


This thesis investigates relationships between extensions 
in technology and both cost and schedule slippages in the 
development and production phases of weapons system acquisi- 
tion. The primary objective is to determine if the amount of 
technology embodied in a given weapons system can be employed 
to predict a Department of Defense (DOD) contractor's perfor- 
mance in meeting cost and schedule targets. The analysis used 
a sample of 15 U.S. military tactical air-launched munitions 
systems. It begins with a review of the literature regarding 
technology measurement and its connection to cost and schedule 
outcomes. Next, measures of technological progress are 
developed and displayed. Third, the process of creating cost 
growth and schedule slippage measures are discussed. The 
relationships between technological complexity and cost and 
schedule outcomes are then empirically tested. Major findings 
indicate that measures of extension in technology are worth- 


while for explaining production and total program cost growth. 
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I. INTRODUCTION AND LITERATURE REVIEW 


A. BACKGROUND 

A number of prior studies have used data describing 
measures of performance and other properties and 
characteristics as indicators of technology embodied in a 
system. These studies then endeavored to relate these 
measures to contractor cost and schedule performance. This 
thesis is a follow-on to these works. This chapter begins 
with a literature review of the techniques currently available 
involving technology measurement and its relationship with 
cost and schedule outcomes. It will then describe how the 
concepts in those works can be used as the groundwork for the 
analysis conducted in this thesis and discuss the basic 
methodology to be used. This first chapter will conclude with 


a preview of subsequent chapters. 


B. PURPOSE 


The intent of this thesis is to determine what, if any, 


relationship exists between the state of the art in technology 


and advances in technology embodied in weapons systems and 
contractor cost and schedule performance during acquisition of 
these systems. It will attempt to analyze the particular 
relationships between technological sophistication and 


advances in technology of a set of U.S. military air-launched 





tactical munitions systems and a Department of Defense (DOD) 
contractor's ability to achieve control over cost growth and 
schedule slippages in both the development and production 


phases. 


C. LITERATURE REVIEW 

In this section, several key studies are reviewed that 
apply to the objective of this thesis and to the methodology 
to be used in the analysis. That is, the following studies 
examine methods to create technology measures and/or 
procedures to relate the level of technology in a system to 
cost performance. Each of these studies has shown that 
variables reflecting system performance, properties, and 
characteristics can be combined to measure the amount of 
technology embodied in a system. These technology describing 
variables can also be used to measure advances in the 
technology of upgraded/updated systems. These studies require 
the accumulation of numerous technology describing variables 
that distinctly indicate a system's performance, properties, 
and characteristics. Each approach then combines variables 
into a single measure that reflects the quantity of technology 
in a given system. Some of the works have then attempted to 
relate the level of technology to system development and/or 
production costs. A common theme is an understanding that the 


cost of extending technology relates to the magnitude of the 





extension.! (Refs. 1,2,3] The analysis in this thesis builds 


upon the work of the authors reviewed. The following reviews 
are presented in chronological order. 
1. Alexander and Nelson, 1972 
Alexander and Nelson [Ref. 6] use a linear regression 
method to analyze their data. A noteworthy point, applicable 
to the discussion in this thesis, lies in their discussion of 
the use of a graphically displayed, regression trend line to 
identify the average or expected date of technology of systems 
over time. This allowed them to display the level of 
technology embodied in a given system. Points representing 
individual systems that were plotted above the trend line 
indicated that the technology in that system was "ahead of its 
time." Similarly, points below the line indicated that the 
systems were "late" or "conservative" developments. 
2. Dodson, 1977 
In this study, Dodson [Ref. 7] used three technology 
indicators. He ran a multivariate regression with the year of 
development as the dependent variable. The regression 


equation was in the following form: 


¥, = Ky + K,X, + kK2X, + k3X, 


‘Important research by such authors as Dodson & Graver 
(Ref. 4] and Dodson (Ref. 5] are not reviewed here. These two 
studies, among others, made use of ellipsoidal hypersurfaces 
to represent levels of technology. They represent important 
contributions to the field of technology measurement, but are 
not directly relevant to the focus of this analysis. 














where: 

Y,= The Year of Technology (defined as the year 
that the system should have been introduced 
given the technology embodied in the system) 

k, = The regression coefficients 


X, = The technology indicators. 


He calculated Y, for each system in the sample and postulated 
that if the actual year of development was less than Y,, then 
the system was “ahead of its time." Similarly, if the actual 
year of development was greater than Y,, then the system was 
technologically "behind the times" when it was introduced. 
Therefore, Y, - Yactuas WaS a measure of the technological 
advance represented by each system in the sample. 

In an effort to associate costs with technology, 
Dodson next conducted another regression with cost as the 
dependent variable and technology advance (¥, — Yactuar) aS an 
independent variable. He estimated both Research §& 
Development (R&D) costs and procurement costs in separate 
regressions. Procurement cost was approximated from a cost 
estimating relationship (CER) incorporating the Year of 
Technology. This was necessary to compensate for the changes 
in production technology over time. Dodson reasoned that an 
advanced design would be relatively more expensive to produce 
with older manufacturing technology. At the same time, 
advanced manufacturing technology should result in a drop in 


procurement cost. He found that his estimates for R&D costs 











were not completely satisfactory, however. This was largely 
due, he explained, to certain unidentified costs of R&D that 
were not included in the regression model. Dodson had 
somewhat better success with procurement costs. 
3. Gordon and Munson, 1981? 
This paper examined technology extension and its 


measurement using two types of equations [Ref. 8]: 


S = KV, + K,V, +... + KV, 
and 
S = V,[K.V, + K,V; +... + K,Vy) 
where: 
Ss = State of the Art (the current extent of 
technology) 
K, = A judgmental or statistical weight 
Vv, = The value of the ith technology describing 
variable. 


The first is a simple linear equation combining weighted 
characteristics. The second equation is a multiplicative form 
that is intended to be used when one variable must be present. 

An important new contribution made by Gordon and 
Munson in this paper was to propose factor analysis as a means 


to collect a large number of technology describing variables 


2Discussion adapted from review in Greer [Ref. l:pp. 11- 
13) 


into groups that have similar relation to or influence on the 
state of the art in technology. The advantage of factor 
analysis is that it allows for a large number of variables to 
be incorporated into the model by aggregating several 
correlated variables into a few, more manageable, composite 
measures called factors. The factor scores can then be 
employed for further analysis. Factor analysis represents yet 
another tool for relating technology and cost measures. 
4. Knight, 1985 

Two important contributions were made in this study 
{Ref. 9]. First, Knight distinguished between functional and 
structural technology measures and related the two. He 
defined a functional measure of technology as: "The 
capability of each system to perform its intended tasks." 
[Ref. 9:p.107] Knight also studied functional technology 
measures over time. This study provided guidelines for data 
collection. 

5. Greer, 1988 

These previous studies examined techniques to define 
extensions in technology, usually with the use of regression 
analysis as a common thread. Technology describing variables 
were typically combined to produce single measures represent- 
ing the amount of technology in a system. The "Year of 
Technology" was usually employed as the dependent variable in 
the regression equation. Several studies then attempted to 


relate the scale of technology to system costs. 


6 











Building on these works, Greer [{Ref. 1] undertook to 
develop a cost estimating model, using technolory measurement 
methods, that could both predict costs and be useful for cost 
control purposes. Using technology measures, he first 
developed a time regression to predict the amount of time 
required for development of a system. This predicted time was 
then used in a cost regression calculating an "ex ante" cost 
estimate for each system. 

Next, the actual time for the project was compared to 
the predicted time to calculate a residual (similar to 
Dodson's Y, — Yactuas COMputation). The cost regression was run 
again to produce a new cost estimate based on the residual 
project time. The difference between the ex ante cost 
estimate and the cost estimate based on actual time was called 
the “Variance Due to Time." This represents that portion of 
variance explained by time delays. Greer also compared actual 
cost to the cost estimate based on actual time to ascertain a 
"Cost Control Variance." This variance revealed the quality 
of a project's cost control. 

In developing the two variance measures, Greer hoped 
to explain the causes of cost variance for each system he 
analyzed. The results indicated whether observed variances 
were due to time or to cost control. In order to control 


costs, he concluded, it was essential to explain the variance 


between predicted and actual costs. 





Greer also noted that development cost is not a smooth 
function of time. "If a program drags on beyond its intended 
completion date, it becomes relatively more costly to compress 


the required accomplishment into an increasingly abbreviated 


time horizon." [Ref. l:p. vi] 
6. Moses, 1989 


In this study, Moses [Ref. 3] revised and extended 
Greer's work. He employed the "Year of Technology" approach 
(the focus in this thesis) to technology measurement as did 
Alexander & Nelson and Dodson rather than Greer's use of the 
ellipsoid method. 

Like Greer, Moses addressed the relationship between 
technology and development costs and created methods for 
determining development cost variances. In an important new 
thrust, the report extends the analysis to examine production 
costs as well as development costs. Moses used data for 18 
satellite systems to empirically test this approach to 


technology measurement and cost prediction and control. 


D. MEASURING EXTENSIONS OF TECHNOLOGY 

The studies reviewed here focus on methods to measure 
technology. One commonly used method is the "Year of 
Technology" Regression approach. This procedure uses multiple 
regression to relate several technology describing variables 


and time in the following general form: 





Y = Ky) + K,X, + KX, +... + KX, + e 


where: 


Y = The actual year that the system was produced 
or became operational 


K, = Regression coefficients 
X, = Technology describing variables 
e = Residual. 


Values predicted from the regression equation for a given 
system represent the "Year of Technology" for that system. As 
Dodson noted, if the actual year that the system was produced 
or became operational is less than the calculated year of 
technology, then the system represents an advancement in 
technology. 

According to Moses, the essence of the Year of 
Technology method is to combine a number of technology 
variables into summary measures and express them in terms of 
time (years). Using this approach, he created three 
technology measures for each system in his sample: STAND, 
ADVANCE, and REACH. These are briefly defined as follows: 

- STAND--The actual year that the system becomes 
operational which represents, in general, where 
technology currently "stands." Stand equals "Y" in 
Dodson's equation Y¥, - Y. For this thesis, STAND is equal 
to Full-Scale Development (FSD) Start Date. 
ADVANCE--The extension of technology beyond the current 
state of the art. This measure is the deviation from the 


trend line discussed by Alexander & Nelson. In Dodson's 
equation, ADVANCE = Y, - Y. 











- REACH--A measure of the level of technology embodied in 
a system equal to Dodson's "yY,." 


This thesis will use STAND, ADVANCE, and REACH measures to 
summarize the technology associated with a particular weapon 
system. These measures will then be used in an attempt to 
explain cost and schedule outcomes. 

A related work by Lienhard [Ref. 10]° supports the 
"Year of Technology" approach. In examining several 
technologies over time, Lienhard studied the rate that 
technology improves and whether this rate changes over time. 
He noted that, once established, the rate of improvement of a 
given technology does not change. If so, attempts to 
accelerate advances in technology "ahead of their time" could 
be extremely costly. Therefore, the Year of Technology 
approach appears sound and it is this approach that will be 


used in this analysis. 


E. BENEFITS 

Building on the techniques established in the literature, 
this analysis will examine a sample of several U.S. military 
tactical air-launched munitions systems. It will contribute 
to the existing body of knowledge by empirically analyzing 
these systems using the methods described earlier. It seeks 
to estimate and explain a government contractor's tendency 


towards cost growth and schedule growth resulting from 


3Discussion adapted from Greer [Ref. 1:p. 38]. 


10 








advances in and extensions to these high-technology weapons 


systems. 


F. OBJECTIVES 

The objective of this thesis is to determine if there is 
a relationship between the state of technology in air-launched 
tactical munitions and a contractor's performance in meeting 
cost and schedule targets. Using the techniques reviewed in 
the literature, the following will be undertaken: 

- A statement and discussion of the hypothesis that a 
relationship may exist between system technology and 
contractor cost and schedule performance. 

- Development of indicators of technology for sir-launched 
tactical munitions that can be employed to assess a 
contractor's ability to meet cost and schedule targets. 

- Using a "Year of Technology" analysis on tactical air- 
launched munitions data, construct summary technology 


measures. 


- Examine, discuss, and display procedures for creating 
measures of cost and schedule growth. 


- Perform a statistical analysis of the hypothesis and 
review results and findings. 


G. SUBSEQUENT CHAPTERS 
1. Chapter IT 
This chapter will begin with a description of the 
origin of the sample and the data sources for technology 
measures. Following techniques demonstrated by Moses [Ref. 
3}, technology measures for tactical air-launched munitions 


will be created using indicators of: 


11 





Speed 

Range 

Guidance Systems 

Sensors 

- System Reliability. 

Next, data sources for cost and schedule outcomes will be 
discussed. The chapter will conclude by defining the 
hypothesis. The hypothesis is based on how three measures of 
technology: stand, advance, and reach, relate to cost growth 
and schedule slippages. 

2. Chapter IIT 

This chapter will examine the process of creating 

technology measures and will include the "Year of Technology" 
regression analysis and a display of measures. It will also 
examine the process of developing cost and schedule measures 
through a discussion and review of procedures and data in the 
study by Tyson et al. (Ref. 11].* It will address controls 
and contain a display of measures. 

3. Chapter IV 

Chapter IV will provide the statistical analysis of 

the hypothesis introduced in Chapter II. It will test the 
relationship of cost and schedule growth, during both 


development and production phases (as the dependent 


“For brevity's sake, this 1989 study by Tyson, Nelson, Om, 
and Palmer is hereinafter referred to simply as the "Tyson 
study." 





variables), and the level of system technology (reach, stand, 
and advance as the independent variables) in a regression 
analysis. Included will be a discussion and display of 
results. 
4. Chapter V 
This final chapter will summarize the key aspects of 


the analysis and present the conclusions of the study. 
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II. SAMPLE, DATA, AND HYPOTHESIS 


This chapter is organized into four sections. The first 
presents the tactical air-launched munitions systems to be 
used as the sample in this thesis. The second and third 
sections discuss the data and its sources. Technology data 
are addressed first and are followed by cost and schedule 
information. With an understanding of the sample and data to 
be used in this analysis, the final section of this chapter 


will introduce the basic hypotheses. 


A. SAMPLE 

The sample consists of 15 tactical air-launched munitions 
systems used in the Tyson study [Ref. l11l:see p. III-2]. 
Tyson's original sample consisted of 16 systems. One, the 
AGM-53A Condor, retired circa 1979, was omitted in this thesis 


due to a lack of available technology measures. Table II-1 is 


a list of the sample systems and their Full-Scale Development 


(FSD) start dates. 

These 15 systems represent three distinct subsets of 
missile types with unique mission requirements: (1) Air-to- 
air (anti-air), (2) Air-to-surface (tactical air-launched), 
and (3) Air-to-Surface (helicopter launched). As will be seen 
in Chapter III, these mission differences will have an impact 


on the variable selection process. Tyson's data identifies 





Table II-1 


SAMPLE TACTICAL AIR-LAUNCHED MUNITIONS SYSTEMS 





OBS Name Designation PSDSD 
1 Sparrow AIM-7E 1960 
2 Sparrow AIM-7F 1966 
3 Sparrow AIM-7M 1978 
4 Sidewinder AIM-9L 1971 
5 Sidewinder AIM-9M 1976 
6 Phoenix AIM-54A 1962 
7 Phoenix AIM-54C 1977 
8 HARM AGM-88A 1978 
9 Harpoon AGM-84A 1973 

10 Maverick AGM-65A/B 1968 

11 Maverick AGM-65D/G 1976 

12 AMRAAM AIM-120A 1982 

13 Hellfire AGM~-114A 1976 

14 TOW BGM-71A/B 1963 

15 TOW2 BGM-71D 1978 


the missile systems by mission, design, and series using 


standard missile system designation. A brief explanation of 


the missile designation structure is provided in Appendix A. 
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There are a variety of air-launched munitions that are not 
missiles--gravity bombs are one such example. However, this 
sample consists solely of missiles. Therefore, for the 
purposes of this thesis, the terms missile and munitions are 
used interchangeably. 

The sample is spread over 22 years when grouped by FSD 
start date. All of the programs are still in production or in 
service with the U.S. Navy, Army, Air Force, and/or Marines or 
in foreign service. Where a successor system of a particular 
missile resulted in a design that was technologically 
indistinguishable from its predecessor, Tyson combined systems 
into a single program. Thus, for example, the AGM-65A and 


AGM-65B versions become the AGM-65A/B. 


B. TECHNOLOGY DATA 
Data, consisting of numerical values for the technology 
measures to be considered in the analysis, were obtained from 
a variety of sources [Refs. 12-17 and additional classified 
sources].* Prior to actual data collection, it was necessary 
to select the variables to be considered in this analysis. 
The selection of measures describing the amount of 
technology embodied in each system required certain pre- 
considerations. According to Dodson and Graver, technology 
The author is indebted to the Naval Postgraduate School 
and, in particular, Dr. Ball of the Department of Aeronautical 
Engineering for help in developing technology measures. 


Invalvable assistance was also provided by MAJ Paul F. 
O'Sullivan, Jr., USA and MAJ Michael Staggs, USA. 
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describing properties must be, at least to some extent, a 
consequence of engineering development rather than a by- 
product of that process. The variables selected should be 
design goals of system development. Ideally, these variables 
should be arranged so that increasing values display greater 
technological sophistication. Additionally, the variables 
selected should be those whose values are determinable in the 
early stages of a system's life-cycle so that they may be used 
for predicting future cost and schedule performance [{Ref. 4: 
pp. 13-14]. 

Technical expertise was sought tc develop the list of 
technology measures. A listing of candidate variables 
describing the technical characteristics, properties, and 
performance criteria of the tactical air-launched munitions 
systems was developed. This candidate list is provided in 
Appendix B. 

Some of these characteristics, properties, and performance 
measures were considered to be simply by-products of design or 
were not stated in a form that could reveal much about 
technological sophistication. These measures were dropped 
from further consideration. The list of candidates was 
furtner reduced by data availability. Due to the variety of 
missions represented in the sample, and due to programmatic 
and record-keeping differences among the military services, 
many of the candidate measures are not used or maintained for 


individual systems. The result of the data collection and 
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review process was the identification of 14 variables that 


could be used to describe the technology embodied in the 
missile systems. These variables are listed in Table II-2. 
Values were obtained for each of the 14 variables, with 
one exception: MSR was not available for the AIM-120A AMRAAM, 
the newest system in the sample. The mean value of the 
Sparrow (AIM-7 series)--the system with the closest, similar 
mission (intermediate range air-to-air)--was inserted to 
preclude a distortion of the data to be used in later portions 
of the analysis.” With 14 variables to describe 15 systems, 
further variable reduction was necessary. This process will 


be discussed in Chapter III. 


C. COST AND SCHEDULE DATA 

This thesis employs cost and schedule data from the Tyson 
study (Ref. 11]. In this work, Tyson, et al., explored trends 
in cost and schedule outcomes of a variety of major 
acquisition programs. They also sought to determine the 
effectiveness of initiatives to improve these outcomes. The 
study examined cost growth and schedule slippages by equipment 
type (including tactical air-launched munitions), by time 
periods, by development and production phases, and by 


development type (new starts or major modifications). The 


*Since portions of the variables or their values are 
classified, they will not be displayed here. As will be seen, 
only regression results are presented which are _ not 
compromisable. 
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TABLE II-2 


VARIABLES DESCRIBING 


TACTICAL AIR-LAUNCHE MUNITIONS TECHNOLOGY 


Range (RNG) Variables: 


RNG1 
RNG2 


Altitude 
ALT1 
ALT2 


- Maximum Missile Range (Meters) 
~- Minimum Missile Range (Meters) 


(ALT) Variables: 
- Maximum Missile Altitude (Meters) 
- Minimum Missile Altitude (Meters) 


Speed (SPD) Variables: 


SPD1 
SPD2 
SPD3 


Terminal 
TGS1 


TGS2 


ECCM 


Platform 


IMCC 


DSP 


- Missile Speed (Mach) 
- Maximum Platform Launch Speed (Mach) 
- Minimum Platform Launch Speed (Mach) 


Guidance System (TGS) Variables: 
- TGS Redundancy 
Coded: 0 Single System 
1 Dual Systems 
2 Multiple Sv~*ems 
- TGS Reliability 
Coded: 0, 1, 5+ 2 based on expert assessment 
of system reliability (classified). 
- Imvroved Electronic Counter Measures Defeating 
Capabilicy 
Coded: 0 No 
1 Yes 


Sensor Variables: 
- Inertial Mid-Course Correction Capability 
Coded: No 
= Yes 
- Digital Signal Processing Capability 


Missile Fuzing Variable: 


FUZE 


- Fuze Type 
Coded: 0O or 1 based, as above, on expert 
assessment of system reliability 
(classified). 


Missile System Reliability (MSR) Variable: 


MSR 


- Percent Reliability, Given a Successful Launch 
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reason for interest in the Ilyson study is its collection of 
cost and schedule histories for the sample of tactical air- 
launched munitions. 

The Tyson study obtained cost and schedule information 
primarily from Selected Acquisition Report (SAR) data. 
Supplemental information was obtained from Development Concept 
(DCP) statistics and other data from the military services, 
Department of Defense (DOD) acquisition/program offices, and 
defense industry sources [Ref. 11, p. III-3}. 

Measures of outcome were based on: 

- COST GROWTH--Development, Production, ard Total: An 
indicator of good program performance is the ability of 
the contractor to develop and produce the _ system 
according to the cost target. 

- SCHEDULE SLIPPAGE--Development and Production: As 
above, an indicator of good performance is the degree to 
which a system can be developed and produced within 
planned time frames. 

The study produced measures for five program outcomes. These 
are listed in Table II-3.% They used the term "growth" in 
cost and schedule since most program changes reflected 
increases in program cost or time. The study found that, in 


a few cases, such as when development or production quantities 


were reduced, cost and schedule "growth" was negative. The 


*They also examined quantity changes which "give clues to 
such issues as reasonableness of the development plan, the 
degree of producticn stability, and the prevalence of program 
stretchout." [Ref. 1l:p. III-5] We restrict our discussion in 
this thesis to cost and schedule outcomes. 


20 








TABLE II-3 


PROGRAM OUTCOMES 


OUTCOME 


Development Cost Growth 


Production Cost Growth 


Total Program Cost Growth 


Development Schedule Growth 


Production Schedule Growth 





process used for calculating program outcome measures will be 


discussed in Chapter III. 


D. HYPOTHESES 

This section discusses the proposed associations between 
a government contractor's performance in meeting cost targets 
and planned schedules and the technological complexity 
embodied in a weapons system. The purpose of the study is to 
determine if a relationship exists between technology 
extensions and cost growth and between technology extensions 
and schedule growth in both development and production phases 
of weapons system acquisition. The objective here is to 


hypothesize why such associations may exist. 
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The hypotheses, then, are based on how three measures of 
technology introduced in Chapter I, REACH, ADVANCE, and STAND, 
relate to cost growth and =xchedule slippages. There are five 
distinct areas to be examined: three involving cost growth 
and two involving schedule growth. 

1. Technological Complexity and Cost Growth 

The first three hypotheses rest on the idea that there 
is a direct relationship between cost growth of a given system 
and the technological complexity of the system. Two variables 
describe the technology associated with a given system: STAND 
and ADVANCE. STAND measures the current state of technology 
at the time of system development. This time is defined as 
the year of Full-Scale Development for each system. Thus, 
STAND measures the general state-of-the-art of missile 
technology at the point when each individual system is 
developed. It is hypothesized that costs become more 
difficult to control as the degree of technological complexity 
increases. Hence, cost growth is expected to be positively 
associated with STAND. 

The second variable is ADVANCE, which measures the 
incremental change in technology achieved by the development 
program. Thus, ADVANCE measures the incremental jump in 
technology, beyond the prevailing state-of-the-art, associated 
with the specific system being created. It is hypothesized 


that costs become more difficult to control as the incremental 
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advance increases. Hence, cost growth is expected to be 
positively associated with ADVANCE.‘ 

Three measures of cost growth are examined. To 
understand the differences between acquisition program phases, 
development and production cost growth, representing two 
distinct phases of DOD's weapons system acquisition strategy, 
are separated. Thus, development cost growth and production 
cost growth are two measures. The third is total program cost 
growth which is the total growth in costs for both the 
development and production phases. Although production costs 
are a much higher percentage of the total than development 
costs, development costs are still important because most 
technical difficulties are encountered in the development 
phase (Ref. 1l:p. III-7]. The hypotheses involving 


cost growth are: 


H,: Development Cost Growth = f£{STAND, ADVANCE) 
H,: Production Cost Growth = £{STAND, ADVANCE} 
H,;: Total Program Cost Growth = f{STAND, ADVANCE} 


2. Technological Complexity and Schedule Slippages 
The fourth and fifth hypotheses concern time, measured 


as an increase (or decrease) of the schedule beyond the 


“Recall that REACH was the third variable. Since REACH 
is the sum of STAND and ADVANCE there is no additional 
information and so, it is excluded from the analysis. 
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original estimate. Hypotheses concerning schedule slippage 
are analogous to those for cost growth. It is hypothesized 
that schedules become more difficult to meet as the degree of 
technological complexity increases. Hence, schedule growth is 
expected to be positively associated with STAND. It is also 
hypothesized that schedules become more difficult to control 
as the incremental technological advance represented by the 
particular system increases. Hence, schedule growth is 
expected to be positively associated with ADVANCE. 

Two measures of schedule growth are examined. As with 
cost growth, development and production phases are separately 
viewed. Total schedule growth is not relevant since the 
various development and production contracts of a given 
program usually require dissimilar and unrelated scheduling 


commitments. The hypotheses for schedule growth are: 


H,: Development Schedule Growth £{STAND, ADVANCE } 


H,: Production Schedule Growth f { STAND, ADVANCE) 


SUMMARY 

This chapter has examined the sample to be analyzed and 
the sources of data. Using this information, the next step is 
to develop technology, cost, and schedule measures to be used 


to test the hypotheses presented in this chapter. Chapter III 


will discuss the process of creating summary technology 








measures and will provide a review of Tyson's procedures for 
creating cost and schedule measures. Chapter IV will then 


test the hypotheses. 

















III. TECHNOLOGY, COST, AND SCHEDULE MEASUREMENT 


In order to test the hypotheses presented in the previous 
chapter, summary measures of technology, cost, and schedule 
are required. The process of creating these measures is the 
next step. Technology measures will be addressed first, 


followed by cost and schedule measures. 


A. CREATING TECHNOLOGY MEASURES 
In Chapter II, data sources were introduced and the 
initial selection of technology describing variables was 
discussed. In this section, the process of creating measures 
of technology is continued. The first step in this process is 
variable reduction. The second step is the creation of 
summary measures of technology. 
1. Variable Reduction 
Recall from Chapter II that data were obtained on 14 
variables for the 15 systems in the sample. Since this many 
variables, given a sample size of only 15, is statistically 
unmanageable, further variable reduction was necessary. Three 
methods of variable reduction were chosen. The first two used 
statistical methods. The third method involved a judgmental 
process based on technical expertise. 
The first method of variable reduction used a 


univariate regression. This method rests on the assumption 
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that technology should increase with time and that good 
technology variables should show a positive relationship to 
time. Thus, a time regression is used to identify the best 
individual variables. The year of development, which was 
previously defined as the FSD start date, was the dependent 
variable in a series of regression equations. Each of the 14 
technology variables was used in a separate regression as a 
Single independent variable. 

Some results of this initial series of regressions 
were unexpected. That is, coefficients of several of the 
variables were negative, when they were expected to be 
positive. This could signify that these variables were not 
the indicators of technology that the technical experts 
thought them to be. Since, upon consultation, they thought 
this unlikely, a review of the data values was undertaken. 
Three categories of the variables describing missile 
performance--RNG, ALT, and MSR--were found to be affected by 
mission requirements. Recall from the description of the 
sample that there are, at least, three distinct groups of 
tactical air-launched munitions in the 15 system sample: (1) 
air-to-air, (2) air-to-surface (tactical air-launched), and 
(3) air-to-surface (helo launched). It was determined that 
design considerations of a given system dictated certain 
performance parameters. This made direct comparisons among 
the three different groups difficult, if not impossible. 


Restricting the sample to just AIM or AGM series missiles was 
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not considered an acceptable alternative since that would lead 
to still more problems due to an increasingly small sample 
size (n < 15). 

To make the RNG, ALT, and MSR variables comparable 
amongst all three groups, data transformation was necessary to 
normalize the measures by group. Actual data values were 
converted to ratios. To accomplish this, some additional data 
collection was necessary. Actual values were obtained for the 
earliest, operational version of each missile system, if not 
already in the sample. Thus, for the AIM-7 Sparrow and the 
AIM-9 Sidewinder, the "A" or "B" series values, representing 
the earliest date of technology, were included to compute the 
ratios. A value of "i" was assigned to the lowest value, 
representing the most primitive level of technology, in each 
of the three mission groups. With the original data value of 
this system in the denominator and each of the remaining 
system values, in turn, as the numerator, the ratios were then 
computed for each of the systems in the sample. In short, 
values for RNG, ALT, and MSR were normalized by dividing a 
group-specific baseline value for RNG, ALT, ana MSR 
respectively. Once the variables were re-configured into 
ratios, the individual univariate regressions were run again; 
this time with much better results. 

The next step was to establish criteria for selection 


of the most significant variables. Upon review of the output, 
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the criteria chosen was an R* greater than or equal to .10 and 
a P-Value less than or equal to .15. Five of the 14 variables 
met these criteria. They are listed in Table III-1. This 


array of variables was labelled Version A. 


TABLE III-1 
VARIABLE REDUCTION: UNIVARIATE REGRESSION RESULTS, 
VERSION A 


Variable p-Value 


ALT2 . -136 
TGS2 . -107 
ECCM -021 
TMCC -111 


FUZE -100 


The second method of variable reduction used was 
stepwise regression. Again the dependent variable was FSD 
date. Also, again, the objective was to identify a small set 
of variables most highly associated with time. Use of the 


normalized data obtained via the ratio transformation was 


continued in this process. The advantage of stepwise 


regression is that it provides an ability to isolate a subset 


of predictor (independent) variables that yield an optimal 





equation. The stepwise method selects from a large set of 
candidates those independent variables that best predict the 
dependent variable (Refs. 22, 23]. Both forward inclusion and 
backward elimination routines were used. The best results 
were obtained with forward inclusion. Three variables were 
selected in this manner--ALT2, SPD3, and ECCM--with a 
relatively high R* of 64.96%. Results of this stepwise 
regression are contained in Table III-2 and is labelled 


Version B. 


TABLE III-2 


VARIABLE REDUCTION: STEPWISE REGRESSION, 
VERSION B 


Stepwise regression of FSDSD on 14 predictors, with n 


Step 1 2 3 
Constant 69.50 71.74 69.07 


ECCM 10.00 
T-Ratio 3.39 


SPD3 11.20 
T-Ratio 2.04 


ALT2 
T-Ratio 


R2 
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Since neither the univariate nor the stepwise methods 


were completely satisfactory for use in the Year-of-Technology 
analysis to follow, it was decided that the variables selected 
by both methods should be used and to add a third list of 
variables. This third list was derived using a judgmental 
approach based on the opinion of technical experts as to those 
variables that should best describe the level of technology in 
these systems. Several combinations of variables developed by 
the experts were tested by both univariate and stepwise 
regression to ensure that they were statistically sound. 
Combinations that contained the "RNG" and "MSR" variables 
achieved unexpected results in the form of negative 
coefficients, indicating a non-positive relationship to the 
passage of time, so were dropped from further consideration. 
A final version, acceptable to the experts and statistically 
sound, was eventually settled on. The variables selected in 
this manner are listed in Table III-3. This third version was 
labelled Version C. 
2. Year of Technology 

With three reduced sets of technology variables now 
selected, the next step was to create summary measures that 
describe the extension in technology of each missile system. 
The process used was the Year of Technology approach. Recall 
that the Year of Technology approach relates technology 
measures to time to predict the expected year that the system 


should be produced. The difference between the actual year, 
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TABLE III-3 


VARIABLE REDUCTION: 
JUDGMENTAL METHOD, 
VERSION C 


ALT2 
SPD3 
TGS2 





which in our case is the FSD start date, and the expected or 
computed year is used as the measure of the advance in 
technology for the individual system. 

Following this process, the years of FSD start of each 
missile program were separately regressed against the 
variables selected in Versions A, B, and C. Results are 
contained in Appendix C. Models B and C explain a moderately 
high portion of the variance (adjusted R? = 55.4% and 51.9%, 
respectively). Version A explains significantly less 
(adjusted R? = 24.2%) due in part to the number of predictor 
variables in the model. Each coefficient is positive which is 
consistent with the values of the technology describing 
variables reflecting increasing technology through time. This 
furnishes some confirmation that the variables selected in the 
previous step appropriately reflect the extent of technology 


and technology growth. 





Tables III-4, III-5, and III-6 are plots of the Year 
of Full-Scale Development (Y) versus Year of Technology (Y,). 
Values for STAND, ADVANCE, and REACH reflecting technological 
complexity or extension were then computed for each system in 
each of the three versions. Results are contained in Table 


IrIm-7.? 


B. COST AND SCHEDULE MEASURES 
This section is primarily a synopsis of Tyson's creation 
of measures for program cost and schedule growth outcomes 
(Ref. 11]. As discussed earlier, Tyson separated cost and 
schedule growth into development and production phases. 
Separate, though similar, procedures were used for cost and 
schedule measurement. 
1. Cost Growth Measures 
The study created cost growth ratios using the 
following procedures and controls: 
- Program cost estimates were collected for each system. 
A base-year, constant dollar figure was used so that 


inflation would not distort comparisons among programs 
that were established at different times. 


‘There are other methods for determining measures of 
STAND, ADVANCE, and REACH. An alternative that Dr. Moses 
tested involved designating an “individual system as a 


reference point. Candidates might be a) an immediate 
predecessor system or b) the predecessor system with the 
greatest REACH (maximum predecessor technology). The 


technology embodied in either reference system would 
constitute STAND, and ADVANCE would be measured as deviations 
from the specific reference system...These alternatives were 
explored with no material enhancement of the analysis." ([Ref. 
2:p. 17£fn] 
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TABLE Ill-4 
YEAR OF TECHNOLOGY PLOT 


85 


80 


75 


70 


65 


60 























VERSION A 
Year of Technology 
He ye 
REACH y va 
Z Y 
ADVANCE Hee 
We | 
saree atta + +/4 og 
i | ? + | 
: i | 
ae ie 
L Fe a 
74 
L fe : STAND 
oe 
Cian ae a 
55 60 65 70 75 80 85 


Year of Full-Scale Development 


34 








TABLE Ill-5 
YEAR OF TECHNOLOGY PLOT 
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TABLE IlI-6 
YEAR OF TECHNOLOGY PLOT 
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TABLE III-7 


TECHNOLOGY MEASURES 


SYSTEM STAND A-ADV A-REACH B-ADV B-REACH C-ADV C-REACH 











AIM-7E 60 5.83 65.83 61.39 -0.74 59.26 


















AIM-7F 66 -0.55 65.45 66.23 -1.26 64.74 







AIM-7M 78 73.86 81.70 78.57 








AIM-9L 71 72.15 71.74 72.46 















AIM-9M 76 77.04 76.70 74.22 








AIM-54A 62 67.84 62.23 63.83 





AIM-54C 77 81.09 77.03 79.86 





AGM-88A 





78 71.02 72.24 74.71 










AGM-84A_ 73 71.81 72.24 72.29 





AGM-65A 
/B 68 -2.55 65.45 4.24 72.24 2.83 70.83 










AGM-65D 
/G 76 3.09 79.09 1.20 77.20 1.21 77.21 










AIM~120A 





77.91 76.70 





79.14 


AGM-114A 71.81 72.24 





72.29 










BGM-71A 
/B 63 8.81 71.81 9.24 72.24 9.29 72.29 






BGM-71D 71.81 





72.24 72.29 


Development costs were obtained for the period from 
program start to initial operating capability (IOC) date. 
Those development costs after IOC were excluded because 
these costs generally consisted of modifications or other 
changes not within the scope of the original development 
project. 

Development Cost Growth (DCG) ratios were then calculated by 

dividing the actual development cost at IOC by the cevelopment 


cost estimate at FSD start using the following formula: 


Development Cost at IOC 


BCG Sa ee 
Development Cost Estimate at FSD 


For production cost growth, there were additional 
considerations. The first was that the data for production 
cost represented the cost to the government, not simply the 
overall cost of the project. Therefore, production cost 
growth ratios more accurately refer to price growth. Second, 
the study took account of changes in quantities procured as 
programs progressed. Adjustments to costs included charges 
for schedule, engineering, inflation, and estimating changes. 
Using a "price-improvement methodology" that included these 
controls, Tyson created price-improvement learning curves to 
calculate the cost of the originally planned quantity. The 
production cost growth (PCG) ratio was based on a comparison 
of the original estimate and the later, revised cost at the 
original planned quantity. The production cost growth formula 


can be stated as: 


2 Production Cost at Production Start 
LOC = eeEeeeeE—eeeeoEeooEeoEoeoOoooEeoooOOOD"DOQ.y 
Adjusted Estimate of Production Cost (Original Qty) 





The final cost outcome measure, Total Program Cost 
Growth (TPCG), was then determined. This ratio was the sum of 
the estimate of development costs at IOC and production cost 
at production start divided by the sum of the development cost 
estimate at FSD plus the adjusted estimate of production costs 


based on the original quantity. The formula was: 


Dev. Cost at IOC + Pr. Cost at Start 


LOC eee 
Dev. Cost at FSD + Adjusted Pr. Cost 


2. Schedule Measures 
Providing measures of schedule slippage was less 
exacting. The formula that Tyson used for development 
schedule growth is simply a measure of time. The ratio was 
computed by dividing the actual span in months’ from 
development start (FSD) to IOC by the original estimate prior 
to FSD. The formula used to compute the development schedule 


growth (DSG) ratio was: 


Actual FSD + IOC (Months) 


DSG = —ooooee ee 
Estimated FSD - IOC (Months) 


Production schedule growth was measured using the same 
technique. The production span was defined as the period from 
contract award to the end of production or, if still in 
production, the last year of planned funding. The production 


schedule growth (PSG) ratio formula is: 


Actual Production Start ~ End (Months) 


psg = —fctuat Production start ~ and (Monetns) _ 
Estimated Production Start ~ End (Months) 
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Actual values of the ratios from the Tyson study for these 


five program outcomes are contained in Table III-8. 


TABLE III-8 
OUTCOME MEASURES 


SYSTEM TPCG 


AIM-7E 
AIM-7F 
AIM-7M 
AIM-9L 
AIM-9M 
AIM-54A 
AIM-54C 
AGM-88A 
AGM-84A 
AGM-65A/B 
AGM-65D/G 
AIM-120A 
AGM-114A 
BGM-71A/B 


BGM-71D 1.70 


* PSG for the AGM-65A/B Maverick was not provided in the 
Tyson study's acquisition program database. The mean value 
of PSG was inserted to preclude a distortion of the data 
for use later in the analysis. 


Source: [Ref. 11:App. A} 
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C. SUMMARY 

This chapter has reviewed Tyson's process of creating 
measures of cost growth and schedule slippage. Using 
techniques from prior studies, measures of technological 
complexity and extension have also been created. With these 
key building blocks in place, the next step is to begin the 


process of examining relationships between technology and cost 


and schedule growth. 








IV. STATISTICAL ANALYSIS AND TESTS 


The purpose of this chapter is to describe the analysis of 
the hypotheses regarding cost and schedule growth and their 
relationship to technological complexity and extension. In 
the previous chapter, techniques developed in previous studies 
were used to create measures of technology. Procedures and 
data used in the Tyson study to create measures of cost and 
schedule growth were also examined. Having developed these 
measures, the main purpose of this thesis is to test for the 
existence of relationships between cost and schedule growth 
and the level of technology embodied in systems. This is to 
be accomplished by statistically testing, using regression 
analysis, the five hypotheses presented in Chapter II. 

This chapter is organized into four sections. The first 
section will discuss methods to examine, evaluate, and prepare 
the variables to be used in the regression analysis. The 
second section will discuss the procedures used in the 
regression analysis itself and will review results. Based on 
these results, the third section will explore some approaches 
to improve upon those results, including the introduction of 
control factors. The final section will summarize the 


results. 
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A. ANALYSIS OF VARIABLES 
Prior to testing the hypotheses, the variables required 
examination to determine their suitability for use in the 
regression analysis. The first step was to check for 
normality and multicollinearity, identify possible outliers in 
the data, and highlight any variables that might need 
transformation {Refs. 23,24}. Several outliers were 
discovered. 
1. Outliers. 
The following alternatives for processing outliers 
were considered. 
a. Disregard Outliers 
Simply put, this means to include the outliers in 
the regression analysis. Since outliers can have a 
significant impact on correlation and regression tests, this 
was rejected. 
b. Discard Outliers 
This procedure eliminates the detrimental impact 
on the tests. This too was rejected, however, since to delete 
these observations would result in substantially reducing the 
sample size. This was unacceptable considering the already 
small number of systems in the sample. 
c. Data Transformation 
There are several methods for transforming data 
values for improved fit. These include square-root, inverse 


function, power, log, exponential, and ratio transformation. 
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However, since the data consisted of a mixture of variables 
(values were, in some cases, negative, some positive, some 
fractional greater than one, and some fractional less than 
one), no one method was mathematically feasible. Therefore, 
this method was also rejected. 

dad. Truncate Outliers 

Truncating outliers involves establishing maximum 
and minimum data values, or boundaries, for each of the 
variables. Outliers are first identified by observation of 
the distribution. Maximum and minimum values are set using 
somewhat subjective methods to define a “reasonable range" for 
the distribution. Values falling above or below this range 
are then truncated to the minimum/maximum boundaries of the 
range. In other words, outliers values are pulled in to form 
a more compact distribution. Since outliers were identified 
for a number of variables, this method was chosen. 
2. Processing Outliers 

The "reasonable range" for outliers was identified by 
reviewing box and stem-and-leaf plots of the variables. Data 
values more than one and one-half times the _ normal 
distribution box-plot length (+ or -) from the extreme values 
of the middle 50% of the data (essentially the data values 
residing at or nearest to the first and third quartiles) were 
selected for truncation. This represented values well outside 


the normal distribution range (approximately 150% greater than 
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or less than the nearest non-outlier value).* The following 
approach was used to assign new values to truncated data 
values. 
a. High Outliers 

DCG for the AIM-7F (refer to Table III-8), by way 
of example, was identified as one of a number of high 
outliers. The highest and second highest non-outliers were 
then identified. The outlier value was truncated at twice the 
percentage difference between the two highest non-outliers. 
The following formula was used to assign truncated values 


[Ref. 24:p. 61]: 


Op = [((Dg - Dp-i/Dp-1) X 2) + 1) x Dy 
where: 

Op = The outlier replacement value 

Dy = The highest non-outlier value 

Dy-1 = The second highest non-outlier value 


The computed value for O, then replaces the original value for 
use in the subsequent regression analysis.? Since this 
procedure resulted in some truncated values still falling 
outside our "reasonable range," two additional controls were 


employed. A subjective limit of 20% was set to limit the 


1fhis method represents the default procedure of outlier 
selection in MINITAB [Ref. 22:p. 210]. 


7Using the DCG example, Dy; = 2.04 and D,., 2.03, then: 


On ((( = (((2.04 - 2.03/2.03) x 2) +1] x 2.04 = 2.06. The 
truncated data value is then established as 2.06. 
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extremes of truncated values. Also, if more than a 10% 
difference existed between the two highest non-outliers, then 
the outlier value was replaced with the lower oi oO, or the 
value 20% larger than the highest non-outlier. 
b. Low Outliers 
Although there were no low outliers were 
identified, they would be processed using a similar technique. 


The following formula was used [Ref. 24:p. 62]: 


Op = [1-( (Dy; - Dp/Dy-1) x 2)] + 1 
where: 

Op = Same as defined above 

D, = The lowest non-outlier 

Di-1 = The second lowest non-outlier. 


As above, the value of O, replaced the original outlier values. 
Similar controls were also used. 
c. Summary 
The outliers that were "pulled in" using these 
procedures were the B-ADVANCE and C-ADVANCE values of the BGM- 
71A/B TOW missile, DCG and DSG of the AIM-7F Sparrow missile, 
and DCG of the AIM-9L Sidewinder missile. With the data 


values for both dependent and independent variables now 


"cleaned," the actual analysis was begun. 











B. TESTS OF HYPOTHESES 

The next step was to test the hypotheses. This was done 
using multiple regression with Tyson's measures of cost and 
schedule growth as single dependent variables. The measures 
of technological complexity and extension (STAND and ADVANCE), 
in three separate versions, were the independent variables. 
The dependent and independent variables are summarized in 
Table IV-1. As explained in Chapter II, REACH was not used in 
this initial series of regressions. Results of the 


regressions are contained in Tables IV-2 through IV-6. 


TABLE IV-1 
TECHNOLOGY AND OUTCOME MEASURES 


TECHNOLOGY OUTCOMES 
(Independent Variables) (Dependent Variables) 


STAND (Current State) Development Cost Growth 


ADVANCE (Extension) Production Cost Growth 


REACH (Amount) Total Program Cost Growth 


Development Schedule Growth 


Production Schedule Growth 
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TABLE IV-2 


DEVELOPMENT COST GROWTH, 
MULTIVARIATE REGRESSIONS 


Dependent Independent Model 
Variable Variable COEFF t p-Value Statistics 


Constant 0.9830 0.54 0.597 
STAND 0.0067 0.27 0.792 
A-ADV -0.0105 -0.29 0.775 


-26 
4.2% 
0.0% 
-774 


Constant 
STAND 
B-ADV 


tout 


Constant 0.506 F 

STAND 0.759 R? 

C-ADV 0.586 Adj.R? 
Pp 
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TABLE IV-3 


PRODUCTION COST GROWTH, 
MULTIVARIATE REGRESSIONS 


Dependent Independent Model 
Variable Variable COEFF t p-Value Statistics 


PCG Constant -0.137 -0.09 0.926 
STAND 0.021 1.08 0.300 
A-ADV 0.051 1.79 0.099 


1.62 
21.3% 
8.2% 
-238 


bououd 


Constant 
STAND 
B-ADV 


Const.ant 0.558 F 

STAND 0.550 R? 

C-ADV 0.118 Adj.R? 
P 
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TABLE IV-4 
TOTAL PROGRAM COST GROWTH, 
MULTIVARIATE REGRESSIONS 


Dependent Independent Model 
Variable Variable COEFF _t p-Value Statistics 
TPCG Constant 0.405 0.28 0.786 F = 1.11 
STAND 0.014 0.70 0.496 R* = 15.6% 
A-ADV 0.041 1.43 0.178 Adj.R? = 1.5% 
Pp = .362 
TPCG Constant 1.4530 1.08 0.303 F = -22 
STAND -0.0003 -0.02 0.987 R? = 3.5% 
B-ADV 0.0196 0.54 0.602 Adj.R* = 0.0% 
P = .807 
TPCG Constant 0.940 0.82 0.431 F = 1.51 
STAND 0.007 0.44 0.666 R? = 20.1% 
C-ADV 0.0679 1.69 0.118 Adj.R? = 6.8% 
P = .260 
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TABLE IV-5 


DEVELOPMENT SCHEDULE COST GROWTH, 
MULTIVARIATE REGRESSIONS 


Dependent Independent Model 
Variable Variable COEFF t p-Value Statistics 
DSG Constant 1.123 0.55 0.595 F = 0.02 
STAND 0.005 0.18 0.858 R? = 0.40% 
A-ADV 0.008 0.04 0.840 Adj.R? = 0.00% 
P = .978 
DSG Constant 0.697 0.40 0.697 F = 0.27 
STAND 0.111 0.46 0.651 R2 = 4.4% 
B-ADV 0.035 0.74 0.474 Adj.R? = 0.0% 
P = .764 
DSG Constant 0.6450 0.40 0.696 F = 0.52 
STAND 0.0121 0.54 0.598 R? = 8.00% 
C-ADV 0.0573 1.02 0.328 Adj.R? = 0.00% 
P = .606 
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TABLE IV-6 






PRODUCTION SCHEDULE COST GROWTH, 
MULTIVARIATE REGRESSIONS 











Dependent Independent Model 
Variable Variable COEFF t p-Value Statistics 









Constant 0.337 0.10 0.921 F = 1.85 
STAND 0.034 0.75 0.470 R? = 25.1% 
A-ADV 0.113 1.74 0.109 Adj.R* = 11.5% 

P = .203 






PSG Constant 2.049 0.74 0.475 F = 1.13 
STAND 0.002 0.06 0.953 R? = 17.1% 

B-ADV 0.103 1.29 0.223 Adj.R? = 2.0% 

P = .357 






PSG Constant 2.19700 0.88 0.396 F = 1.25 
STAND 0.00003 0.00 0.999 R? = 17.2% 

C-ADV 0.12067 1.38 0.191 Adj.R* = 3.5% 

P = .321 











1. Development Cost Growth and Technological Complexity 
Recall that the first hypothesis is that there is a 


direct relationship between cost growth in the development 
phase (DCG) and the scope of the development task measured in 
terms of technological complexity. The regression results are 
mediocre. Coefficients for STAND and/or ADVANCE are negative 
in all three versions which is unexpected. The low values for 
R? and the poor significance levels render this model 
essentially without much meaning. 
2. Production Cost Growth and Technological Complexity 
The second hypothesis is that production cost growth 
(PCG) is directly related to technological complexity. The 
results of these multivariate regressions are somewhat better 
than for DCG. The coefficients are positive as expected and 
the R* values for Versions A and C are much higher. AS a 
whole, however, the models are not significant at traditional 


levels. 


3. Total Program Cost Growth and Technological 
Complexity 


With disappointing results for DCG and PCG, it was not 
surprising that the results obtained for total program cost 
growth (TPCG) were also unimpressive. Except for Version B, 
coefficients were positive as expected. R* values were lower 
than for PCG, but closer to PCG than to DCG. This may reflect 
the greater influence of production costs than development 


costs on total program costs, as put forward by Tyson. 
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Somewhat surprisingly, the R* and adjusted R* values for 
Version C increased. 


4. Development Schedule Growth and Technological 
Complexity 


The fourth hypothesis suggested a direct relationship 
between development schedule growth (DSG) and the STAND and 
ADVANCE measures of technology. Coefficients were positive in 


all three versions, but again, the models were insignificant. 


5. Production Schedule Growth and Technological 
Complexity 


The fifth and final hypothesis is that a direct 
relationship exists between production schedule growth (PSG) 
and technology STAND and ADVANCE. Perhaps the best overall 
results were obtained from these regressions. Coefficients 
were positive in all three versions. The R’*s were relatively 
high compared to the previous models. The highest adjusted R? 
of all of the models was obtained from Version A. Yet again, 


however, the models are not significant at traditional levels. 


C. OTHER CONSIDERATIONS 
1. Additional Tests 
The relatively poor results obtained from the 
regressions suggested a return to the data transformation 
process discussed earlier in this chapter (see Section A.1.c). 
This was done in an etfort to improve the outcomes of the 


regressions. Each of the data transformation techniques was 


tried, in turn and in combination--where mathematically 














feasible--on both the dependent and independent variables. 
Multiple regressions were then rerun on the transformed 
variables. Despite the multitude of regression run in this 
manner, in no case was there a material enhancement of the 
models. 

Univariate regressions were also explored. STAND, 
ADVANCE, and REACH were separately used as single independent 
variables. Regressions were run on the original data as well 
as transformed data in all combinations. Again, no material 
improvement was noted. 

2. Control Factors That Affect Cost Growth 

Since the results to this point were not particularly 
satisfying, a new direction was explored. The Tyson study 
identified several factors that affect cost growth. The most 
significant were program stretch and several acquisition/ 
contracting initiatives such as multi-year procurement, 
competition, prototyping, design-to-cost, total package 
procurement, fixed price development, and contract incentives. 
Tyson's factors are summarized in Appendix D. 

In discussing program stretch, the Tyson. study 
reasoned that DOD and Congress “have sometimes met budgetary 
constraints by stretching out the production schedule, buying 
the same quantity over a longer scheaule, or buying a lessor 
quantity over the same period." (Ref. ll:p. V-1} They 
hypothesized that program stretch, measured as schedule growth 


divided by quantity growth, contributes to cost growth. They 
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performed a series of univariate regressions with cost growth 
--development, production, or total program--as the dependent 
variable and stretch as the independent variable. Their 
findings indicated that program stretch was a strong, positive 
determinant of both production and total program cost growth, 
but not development cost growth. 

Tyson then examined whether acquisition/contracting 
initiatives were associated with lower cost growth. They 
again used univariate regression analysis with cost growth as 
the dependent variable and the acquisition/contract initiative 
as the independent variable. Their findings indicated the 
following relationships: 

- Fixed price development contracts (FPD) were positively 
related and contract incentives for FSD (I-FSD) were 
negatively related to development cost growth (DCG). 

- Total package procurement (TPP) was positively related 
and contract incentives in the production phase (I-PRD) 
were negatively related to production cost growth (PCG). 

- Total package procurement (TPP) was positively related 
while both FSD (I-FSD) and production (I-PRD) contract 
incentives were negatively related to total program cost 
growth (TPCG). 

Tyson's finding suggested new tests. When program 
stretch and acquisition/contracting factors are controlled for 
the technology measures, STAND and ADVANCE could prove useful 


as predictors of cost growth. The following new models, using 


Tyson's variables as controls, were developed’: 


3Actual data values for the control factors are contained 
in Appendix E. 
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Hj: DCG = £{STAND,ADVANCE,I-FSD,FPD} 
H,: PCG = £{STAND,ADVANCE, I-PRD, TPP, STRETCH} 
H,: TPCG = £{STAND, ADVANCE, I-FSD, I-PRD, TPP, STRETCH} 


where: 


I-FSD, FPD, I-PRD, TPP and STRETCH are as defined above. 


Multivariate regressions were performed to test the 
new models. Results are contained in Tables IV-7, IV-8, and 
Iv-9. The DCG models were, once again, unimpressive. The 
best results were obtained for production cost growth and 
total program cost growth using Version C measures of STAND 
and ADVANCE. While the overall models are not significant, 
the t-statistics and p-values for ADVANCE are meaningful. 
This tends to indicate that, when STRETCH and acquisition/ 
contracting factors are controlled, ADVANCE is useful as a 


predictor of production and total program cost growth. 


D. RESULTS 

Low significance levels and the small explanation of 
variability (R?) for the initial models was a common result. 
The low explanatory ability of all the DCG models was not 
entirely unexpected. Both Dodson [Ref. 7] and Moses [Ref. 3] 
used technology variables in an attempt to predict development 
cost (using direct dollar measures of development cost) with 


little success. While they attributed their poor results to 


incomplete modelling, the similarly poor results here with 








TABLE IV-7 


DEVELOPMENT COST GROWTH WITH CONTROLS, 
MULTIVARIATE REGRESSIONS 


Dependent Independent Model 

Variable Variable COEFF t p-Value Statistics 

DCG Constant 0.671 0.32 0.753 F = 0.19 
STAND 0.013 0.03 0.675 R? = 6.9% 
A-ADV -0.013 0.04 0.752 Adj.R? = 0.0% 
I-FSD -0.146 -0.44 0.668 P = .941 
FPD -0.324 -0.50 0.630 

DCG Constant 1.184 0.75 0.473 F = 0.72 
STAND 0.007 0.31 0.764 R? = 22.3% 
B-ADV -0.067 -1.45 0.177 Adj.R? = 0.0% 
I-FSD -0.287 <-0.90 0.387 P = .598 
FPD -0.673 -1.04 0.321 

DCG Constant 0.645 0.39 0.706 F = 0.29 
STAND 0.013 0.56 0.589 R? = 10.3% 
C-ADV -0.040 -0.70 0.499 Adj. R? = 0.0% 
I-FSD -0.197 0.59 0.568 Pp = .879 
FPD -0.418 -0.64 0.538 
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TABLE IV-8 


PRODUCTION COST GROWTH WITH CONTROLS, 
MULTIVARIATE REGRESSIONS 













Dependent Independent Model 
Variable Variable COEFF t p-Value Statistics 





Constant -0.546 -0.27 0.793 F = 1.15 
STAND 0.023 0.92 0.382 R? = 39.0% 
A-ADV 0.039 1.26 0.239 Adj.R? = 5.2% 
I-PRD 0.310 1.16 0.276 P = .401 
TPP -0.172 0.49 0.732 


STRETCH 












Constant 0.469 0.27 0.796 F = 0.97 
STAND 0.011 0.49 0.635 R? = 35.1% 
B-ADV 0.037 0.97 0.356 Adj.R* = 0.0% 
I-PRD 0.251 0.90 0.392 P = .482 
TPP -0.560 -1.11 0.297 


STRETCH 


Constant 





STAND 0.015 0.86 0.414 R? = 55.4% 
C~ADV 0.091 2.34 0.044 Adj.R? = 30.6% 
I-PRD 0.124 0.52 0.617 P = .139 
TPP -0.783 -1.83 0.100 


STRETCH 


TABLE IV-9 


TOTAL PROGRAM COST GROWTH WITH CONTROLS, 
MULTIVARIATE REGRESSIONS 


Dependent Independent Model 
Variable Variable COEFF t p-Value Statistics 
TPCG Constant 0.180 0.09 0.933 F = 0.90 
STAND 0.012 0.51 0.623 R? = 40.4% 
A-ADV 0.044 1.26 0.244 Adj.R? = 0.0% 
I-FSD 0.266 0.87 0.411 P = .537 
I-PRD 0.183 0.54 0.606 
TPP 0.015 0.03 0.977 
STRETCH -0.020 -0.15 0.886 
TPCG Constant 1.161 0.61 0.558 F = 0.73 
STAND 0.001 0.05 0.959 R? = 35.3% 
B-ADV 0.037 0.91 0.389 Adj.R* = 0.0% 
I-FSD 0.161 0.54 0.603 P = .642 
I-PRD 0.194 0.54 0.603 
TPP -0.402 -0.77 0.464 
STRETCH -0.023 -0.16 0.874 
TPCG Constant 1.253 0.86 0.412 F = 2.05 
STAND 0.002 0.09 0.934 R? = 60.6% 
C-ADV 0.105 2.55 0.034 Adj.R* = 31.1% 
I-FSD 0.350 1.42 0.193 P = .170 
I-PRD -0.088 -0.29 0.780 
TPP ~0.679 -1.62 0.145 
STRETCH -0.012 -0.11 0.915 
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development cost growth offer some confirmation that 
technology is not a direct predictor of cost outcomes in the 
development phase. 

The results obtained while using control variables in 
addition to the technology variables to predict cost growth 
were improved, indicating that ADVANCE is potentially a useful 
measure for estimating production and total program cost 
growth. 


Chapter V will summarize the thesis and present 


conclusions based on the results obtained in this chapter. 





V. SUMMARY AND CONCLUSIONS 


A. SUMMARY 

This thesis has explored hypothesized relationships 
between technological complexity and both cost growth and 
schedule growth in the development and production phases of 
weapons system acquisition. A key objective was to develop 
measures of technology, cost growth, and schedule growth that 
would be useful for conducting a statistical analysis of 
possible relationships. 

The first step was to identify technology describing 
properties, characteristics, and performance measures for the 
sample of tactical air-launched munitions. With the 
assistance of technical experts, data was obtained for 14 
variables measuring ranye, speed, altitude, electronics, and 
reliability. To make the number of variables more manageable 
for the subsequent analysis, the set of variables was reduced 
by three separate methods: univariate regression, stepwise 
regression, and a judgmental selection method, labelled 
Versions A, B, and Cc. All subsequent analysis was conducted 
three times, once based on each of the three reduced sets of 
variables. 

Using the Year of Technology Approach established in 
previous studies, the reduced set of technology describing 


variables was employed to determine the level of technology 
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embodied in each missile system. The set of technology 
measures were regressed against the actual year in which the 
systems were developed. The resulting regression model 
produced three measures of technology for each missile system: 


- STAND--The actual year in which the missile system was 
developed which measures the current state of technology. 


- ADVANCE--The extension of technology beyond the current 
state of the art for a given system. 


- REACH--The total level of technology embodied in each 
system. 


Measures of program outcomes developed by Tyson, Nelson, 
Om, and Palmer (Ref. 11] were used as cost growth and schedule 
growth measures. In their study, Tyson et al. created ratio 
measures of cost growth by dividing the actual development or 
production costs by the costs estimated at the beginning of 
the project. Similarly, schedule growth ratios were 
calculated by dividing the actual length of the development or 
production project by the time estimated at the beginning of 
tne project. In both cases, certain controls were introduced 
to ensure accuracy in the ensuing ratios. 

The next step was to statistically test for the existence 
of relationships between cost and schedule growth and 
technological complexity. Each of the measures of cost growth 
and schedule growth were regressed on STAND and ADVANCE. The 
results of these initial regressions were unimpressive. 

In an effort to improve upon these models, data transfor- 


mation techniques were employed. Various combinations of 
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square-root, inverse function, power, log, exponential, and 
ratio transformation of the data were tested without material 
enhancement of the results. Various other univariate 
regression tests were experimented with, but again with no 
improvement of the results noted. 

As a final test, a number of control factors that affect 
cost and schedule growth were introduced. These factors 
included program stretchout, a measure of how government 
attempts to meet budgetary constraints by buying the same 
quantity over a longer period of time or a smaller quantity 
within the original schedule. Other control factors consisted 
largely of acquisition initiatives, including total package 
procurement, fixed price development, and contract incentives 
in the development and production phases. Program stretchout 
and the acquisition initiatives with the strongest affect on 
cost growth were used as additional independent variables in 
a series of multivariate regressions. The results, while 
still not highly significant, nevertheless indicated that 
ADVANCE is useful as a predictor of production and total 


program cost growth. 


B. CONCLUSIONS 

The overall low significance levels and extremely small 
explanation of variability in the models establishes that 
measures of technological complexity and extension are not 


strong predictors of cost or schedule growth in either the 
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development or production phases of tactical air-launched 
munitions acquisition. The results obtained from both 
development cost and development schedule tests offers some 
confirmation that technological complexity is not a direct 
predictor of program outcomes in the development stage. The 
use of control factors as additional predictors of program 
outcomes provided some improvement in the results. This 
indicates that technology measures alone are not strong or 
dominant explainers of cost or schedule growth, but when 
obscuring factors are controlled for, technological extension 
(as measured by ADVANCE) is apparently related to production 
and total program cost growth. 

The results have some limitations. The 15 tactical air- 
launched munitions systems were a small sample. In this small 
sample, three distinct missile groups were merged together. 
Data collection for technology variables was, to some extent, 
confined to those measures common to each of the three groups. 
This suggests that improved technology describing measures 
could be obtained for a more homogenous sample of missiles. 
It is possible, as well, that cost and schedule growth 
measures are not fully comparable between the three missile 
groups. This indicates that improved results may be achieved 
through the use of a larger and more homogenous sample of 


systems. 
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APPENDIX A 


MISSILE DESIGNATION SYSTEM 


A chart describing the Department of Defense (DOD) system 


for identifying missile programs is provided below. 
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APPENDIX B 


PROPERTIES, CHARACTERISTICS, AND PERFORMANCE MEASURES 
DESCRIBING TACTICAL ATR-LAUNCHED MUNITIONS TECHNOLOGY 





~ Acquisition (Seeker Range) 

~ Airframe Structure 

- Attack Azimuth Angle 

- Aerodynamic Performance 

- Boost 

- Chaff Defeating Capability 

- Digital Signal Processing Capability 
- Electronic Counter-Meascures Defeating Capability 
- Flare Defeating Capability 

- Format Target Capability 

- Fuzing Success 

- Fuzing System Performance 

- Guidance Success 

- Imaging (Target/Background Reflections) 
- Inertial Mid-Course Correction 

- Inertial Reference 

- Kill Probability (Pk) 

- Launch Acceptability Regions (LARS) 
- Launch Platform Speed 

- Launch Success 

- Lethality 

- Look-Down, Shoot~Down Capability 

- Mid~Course Guidance Performance 

- Missile Message Accuracy 

- Missile Altitude Capability 

- Missile Range Capability 

- Missile Speed Capability 

~ Multiple Target Capability 
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- No-Escapes Zones 

~- Propulsion Management 

- Propulsion System Performance 
- Sensor Performance 

- Safe Separation 

- Seeker Mode 

- Stability and Control 

- Software Assessment 


- Software Validation and Verification 





- Terminal Guidance System Redundancy 
- Terminal Guidance System Reliability 
- Warhead Performance 

- Weapon Effectiveness 

- Weapon Reliability 

- Weather 


Primary Source: [Refs. 19,20] 








Version A 


APPENDIX C 


YEAR OF TECHNOLOGY REGRESSION ANALYSIS 


The regression equation is: 


FSDSD 


Predictor 


Constant 
ALT2 
TGS2 
ECCM 
IMcc 
FUZE 


s = 


5.972 


65.1 + 0.38 ALT2 + 3.18 TGS2 + 4.90 ECCM + 0.87 IMCC 
+ 2.38 FUZE 


6 


Coef 
5.077 
0.377 
3.180 
4.896 
0.866 
2.381 


R-sq 


Analysis of Variance 


SOURCE DF 
Regression 5 
Error 9 
Total 14 
SOURCE DF 
ALT2 1 
TGS2 1 
ECCM 1 
IMCC 1 
FUZE 1 


Stdev t-ratio 
4.064 16.01 
1.733 0.22 
1.963 1.62 
5.276 0.93 
5.885 0.15 
4.393 0.54 
= 51.3% R-sq(adj) 
Ss MS 
337.90 67.58 
321.03 35.67 
658.93 
SEQ SS 
107.36 
153.69 
65.91 
0.46 
10.48 
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p-val 
0.000 
0.833 
0.140 
0.378 
0.886 
0.601 











Version B 


The regression equation is: 


FSDSD = 69.1 + 3.17 ALT2 + 4.96 ECCM + 20.0 SPD3 

Predictor Coef Stdev t-ratio p-val 
constant 69.066 2.180 31.68 0.000 
ALT2 3.169 1.546 2.05 0.065 
ECCM 4.964 3.580 1.39 0.193 
SPD3 20.017 6.510 3.07 0.011 
s = 4.581 R-sq = 65.0% R-sq(adj) = 55.4% 


Analysis of Variance 


SOURCE DF ss MS F p-val 

Regression 3 428.07 142.69 6.80 0.007 

Error 11 230.87 20.99 

Total 14 658.93 

SOURCE DF SEQ SS 

ALT2 1 107.36 

ECCM 1 122.29 

SPD3 1 198.42 

Unusual Observations 

Obs. ALT2 FSDSD Fit Stdev.Fit Residual St.Resid 
14 1.00 63.00 72.24 1.76 -9.24 -2.18* 


* Denotes an observation with a large standardized residual. 
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Version C 


The regression equation is: 





FSDSD = 68.2 + 20.3 SPD3 + 2.63 ALT2 + 1.77 ECCM 
+ 4.91 IMCC + 0.73 TGS2 + 3.15 FUZE 

Predictor Coef Stdev t-ratio p-val 
Constant 68.203 3.472 19.65 0.000 
SPD3 20.301 8.159 2.49 0.038 
ALT2 2.632 1.651 1.59 0.150 
ECCM 1.767 4.386 0.40 0.698 
IMCCc 4.913 4.961 0.99 0.351 
TGS2 0.728 1.848 0.39 0.704 
fuze 3.149 3.512 0.90 0.396 
s = 4.756 R-sgq = 72.5% R-sq(adj) = 51.9% 


Analysis of Variance 


SOURCE DF Ss MS F p-val 

Regression 6 477.95 79.66 3.52 0.052 

Error 8 180.98 22.62 

Total 14 658.93 

SOURCE DF SEQ SS 

SPD3 1 34.02 

ALT2 1 353.70 

ECCM 1 40.34 

IMCC 1 28.99 

TGS2 1 2.72 

fuze 1 18.18 

Unusual Observations 

Obs. SPD3 FSDSD Fit Stdev.Fit Residual St.Resid 
14 0.000 63.00 72.29 2.23 -9.29 -2.21* 


* Denotes an observation with a large standardized residual. 
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NEW/MOD 


TIME 


PROD-U 


PRO 


STRETCH 


C-PRD 


DTC 
MYP 


FPD 


TPP 


I-FSD 


I-PRD 


Source: 


APPENDIX D 


CONTROL FACTORS THAT AFFECT COST GROWTH 


(Ref. 


New vs. modification programs. 


Time period: programs distinguished by year of 
FSD start--late 1960s, the early 1970s, the 
late 1970s, the 1980s. 


Programs that started production in 1985 or 
before. Only programs with production starts 
before 1985 were considered. 


Prototype programs vs. programs that did not 
use a prototype. 


A measure of program stretchout equal to 
production schedule growth divided by 
production quantity growth. 


Programs that included competition in their 
acauisition/contracting strategy. 


Design-to-Cost contracts. 
Multi-Year Procurement contracts. 


Fixed Price Development contracts. Coded: 0 
without FDP, 1 with FPD. 


Total Package Procurement contracts. Coded: 0 
without TPP, 1 with TPP. 


Contracting incentives in the Full-Scale 
Development phase. Coded: 0 without 
incentives, 1 with incentives. 


Contracting incentives in the production phase. 


Coded: 0 without incentives, 1 with 
incentives. 
11:App. A] 
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APPENDIX E 


COST GROWTH OUTCOME CONTROL MEASURES’ 


SYSTEM I-FSD FPD I-PRD TPP STRETCH 
AIM-7E pe 19) 1 0 9.1471 
AIM-7F 1 0 1 0 1.1627 
AIM-7M 1 0 1 0 1.1667 
AIM-9L 1 0 1 0 2.2439 
AIM-9M 0 0 0 0 1.0749 
AIM-54A (¢) 1 0 0 1.2245 
AIM-54C 1 0 1 0 0.7794 
AGM-88A 1 0 1 6) 1.5333 
AGM-84A 1 0 1 0 3.2105 
AGM-65A/B 0 0) 6) 1 2.1356* 
AGM-65D/G 1 te) 1 0 1.0974 
AIM-120A 0 1 0 0 1.1100 
AGM-114A 1 0) re] 0 1.2020 
BGM-71A/B 0 0 1 6) 3.8475 
BGM-71D 1 0 1 0 1.0562 


*STRETCH FOR the AGM-65A/B Maverick was not provided in the 
Tyson study's acquisition program database. The mean value of 
STRETCH was inserted to preclude a distortion of the data for 
use later in the analysis. 


Source: {Ref. 11:App. Aj] 


‘Refer to Appendix D for an explanation of control 
measures. 
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